High Performance Computing (HPC) systems, sometimes referred to as supercomputers, typically include a plurality of compute nodes each having one or more processing units or cores. More typically, such HPC systems include hundreds or even thousands of cores that may be distributed across a network or placed in a close proximity to one another (e.g., in a computer cluster). Such HPC systems are used for a wide range of computationally intensive applications in various fields, including, without limitation, quantum mechanics, weather forecasting, climate research, oil and gas exploration, molecular modeling and physical simulations.
The multiple compute nodes of a HPC system typically operate independently and periodically output information in a burst output mode. The burst output is typically stored to enterprise level storage architectures. One current storage approach for HPC systems is to provide a storage architecture (e.g., parallel file system) that provides enough bandwidth to sustain a 100% duty cycle burst (i.e., simultaneous burst of all compute nodes). This is to ensure that write bandwidth is available when the compute nodes are ready to dump accumulated computations. This approach provides a brute force solution by using hundreds of, for example, conventional block storage machines behind a parallel file system. One factor requiring the use of such a large number of block storage machines is the limited bandwidth (e.g., maximum data input rate) of the block storage machines. That is, the processor complexes of such storage machines are typically limited to bandwidths of 10 GB/s to 40 GB/s as limited by the current de-facto processor architecture, typically, produced by Intel Corporation. This bandwidth limitation requires large numbers of the storage machines to accommodate the output of an HPC system. The volume of storage machines required to provide the necessary bandwidth causes many infrastructure problems, including management logistics, mean time between failure (MTBF) issues, power infrastructures and cabling.